Skip to content

test: add end-to-end backend regression suite with real ASL fixtures and CI integration#22

Open
aviralsaxena16 wants to merge 3 commits intoOSIPI:mainfrom
aviralsaxena16:output_validation
Open

test: add end-to-end backend regression suite with real ASL fixtures and CI integration#22
aviralsaxena16 wants to merge 3 commits intoOSIPI:mainfrom
aviralsaxena16:output_validation

Conversation

@aviralsaxena16
Copy link

@aviralsaxena16 aviralsaxena16 commented Feb 27, 2026

Hello @1brahimmohamed @jan-petr,

This PR introduces a structured end-to-end regression testing framework for the ASL Method Section Generator backend.

The objective is to improve reliability and establish a deterministic validation pipeline for API behavior and real BIDS-based ASL inputs, in alignment with the GSoC project goals.

All tests pass locally and are validated under CI across multiple Python versions. Snapshot baselines are version-controlled to ensure deterministic behavior across environments.


What This PR Adds

1. End-to-End API Regression Testing

Regression tests are implemented at the FastAPI layer using TestClient to validate:

  • Root endpoint stability
  • /api/report/process/bids behavior
  • /api/report/process/dicom behavior
  • /api/report/report-pdf generation

pytest-regressions is used to snapshot JSON responses, ensuring that future changes do not silently alter report generation logic.

Volatile fields (e.g., nifti_slice_number) are excluded to prevent false-positive regressions.


2. Real ASL Fixture-Based Testing

Added real ASL fixtures under:
tests/fixtures/real_sample/

Includes:

  • sub-Sub1_asl.json
  • sub-Sub1_m0scan.json
  • sub-Sub1_aslcontext.tsv

Tests simulate real BIDS uploads including:

  • JSON metadata
  • ASL context TSV
  • Valid NIfTI file

No internal functions are mocked , the full request lifecycle is exercised:

upload → normalization → metadata extraction → report generation → serialization

This validates:

  • Upload pipeline integrity
  • Metadata normalization logic
  • Report generation consistency
  • Error and warning propagation
  • M0 consistency checks
  • Slice extraction from NIfTI

This approach ensures scientific reproducibility by protecting report text and metadata transformations from unintended drift.


3. Deterministic Sample NIfTI

A minimal valid NIfTI file is included:

app/utils/sample_nifti.nii.gz

This allows CI to exercise nibabel-based slice detection without relying on external datasets while maintaining reproducibility.


4. API Schema & Error Handling Tests

Added coverage for:

  • Missing file validation
  • Invalid DICOM upload handling
  • Correct HTTP status codes
  • Structured error payload behavior

5. PDF Endpoint Validation

Verified:

  • Successful PDF generation
  • Correct application/pdf content-type
  • Non-empty binary response

Minor Fixes Included

  • Renamed modaliy_enum.pymodality_enum.py
  • Updated imports accordingly

Impact

This PR establishes a reproducible testing baseline that enables:

  • Safe refactoring
  • Addition of new ASL variants
  • Future modality expansion (DCE / DSC)
  • Processor-level enhancements
  • Metadata validation improvements

New fixtures can now be added per module with minimal setup.

Test Summary

Screenshot 2026-02-28 031826

(closes #7)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Testing] Integrate Minimal Fixtures for Deterministic Output Validation

1 participant